Evaluation of a mobile application tool (BiliNorm) to improve care for newborns with hyperbilirubinemia in Indonesia

Background Severe hyperbilirubinemia is more frequent in low- and middle-income countries such as Indonesia than in high-income countries. One of the contributing factors might be the lack of adherence to existing guidelines on the diagnosis and treatment of hyperbilirubinemia. We developed a new national guideline for hyperbilirubinemia management in Indonesia. To help healthcare workers use this guideline, a web-based decision support tool application may improve both the adherence to the guideline and the care for infants with hyperbilirubinemia. Methods We developed a web-based application (BiliNorm) to be used on a smartphone that displays the bilirubin level of the patient on the nomogram and advises about the treatment that should be started. Healthcare workers of two teaching hospitals in East Java, Indonesia, were trained on the use of BiliNorm. At 6 months after the introduction, a questionnaire was sent to those who worked with the application enquiring about their experiences. An observational study was conducted in two time epochs. A chart review of infants with hyperbilirubinemia in the two hospitals was sent. The appropriateness of hyperbilirubinemia management during a 6-month period before BiliNorm introduction was compared to that during a 7-month period after its introduction. Results A total of 43 participants filled in the questionnaire, the majority (72%) of them indicated that BiliNorm was well received and easy to use. Moreover, 84% indicated that BiliNorm was helpful for the decision to start phototherapy. Chart review of 255 infants before BiliNorm introduction and that of 181 infants after its introduction indicated that significantly more infants had received treatment according to the guideline (38% vs 51%, p = 0.006). Few infants received phototherapy, but bilirubin level was not measured (14% vs 7%, p = 0.024). There was no difference in the proportion of infants who were over- and under-treated (34% vs 32% and 14% vs 10%, respectively). Conclusions The web-based decision tool BiliNorm appears to be a valuable application. It is easy to use for healthcare workers and helps them adhere to the guideline. It improves the care for infants with hyperbilirubinemia and may help reduce the incidence of severe hyperbilirubinemia in Indonesia.

a1111111111 a1111111111 a1111111111 a1111111111 a1111111111 introduction and that of 181 infants after its introduction indicated that significantly more infants had received treatment according to the guideline (38% vs 51%, p = 0.006). Few infants received phototherapy, but bilirubin level was not measured (14% vs 7%, p = 0.024). There was no difference in the proportion of infants who were over-and under-treated (34% vs 32% and 14% vs 10%, respectively).

Conclusions
The web-based decision tool BiliNorm appears to be a valuable application. It is easy to use for healthcare workers and helps them adhere to the guideline. It improves the care for infants with hyperbilirubinemia and may help reduce the incidence of severe hyperbilirubinemia in Indonesia.

Background
Severe hyperbilirubinemia is frequently observed in newborn infants in developing countries such as Indonesia [1]. Without adequate treatment, this condition can result in bilirubininduced acute and chronic encephalopathy and may even cause neonatal death [2]. Although there are guidelines on the diagnosis and treatment of hyperbilirubinemia to assist healthcare workers involved in the care of newborn infants, such as midwives, general practitioners and paediatricians, a previous research showed that the awareness and adherence to these guidelines is low among midwives and general practitioners in Indonesia [3]. In fact, only 23% of general practitioners and 29% of midwives use these guidelines. In a survey conducted at Dr. Soetomo General District Hospital, Surabaya, Indonesia, it was found that 43% of term babies and 59% of near-term babies received phototherapy (PT), but the total serum bilirubin (TSB) level was below the PT threshold. At the same time, under-treatment was found in 30% of preterm babies. Another study conducted in the USA reported that approximately 60% of paediatricians initiated PT at TSB levels lower than those recommended by the American Academy of Pediatrics (AAP) in newborns aged >72 hours, indicating that paediatricians do not always adhere to guidelines [4]. Over-treatment and under-treatment are problems in hyperbilirubinemia management that can be prevented along with the adherence of healthcare workers to hyperbilirubinemia management guidelines. Low adherence to guidelines by healthcare workers may be due to the difficulty in accessing the available guidelines [3,5]. Among paediatricians in Surabaya, 50% indicated difficulties in obtaining access to a guideline [3]. Therefore, in this study, we developed the first web-based decision application based on the Indonesian hyperbilirubinemia guideline to make it easier for paediatricians, midwives and general practitioners to access and adhere to the guideline. This paper is intended to describe the characteristics of this web-based application, known as BiliNorm [6], and how this application was perceived by paediatricians, midwives and general practitioners. We hypothesised that the introduction of BiliNorm influenced the accuracy of treatment of infants with hyperbilirubinemia and would improve the care for these infants.

Methods
BiliNorm is based on the Indonesian National Guideline on Hyperbilirubinemia [7]. It can be accessed at www.bilinorm.babyhealthsby.org via mobile phone or personal computer with two language options, Indonesian and English [6]. A digital research institute in 2018 estimated approximately 100.000.000 out of total 250.000.000 population in Indonesia uses smartphone [8], where this penetration rate might be higher among healthcare workers population considering them as a more well-educated group. The BiliNorm application algorithm is depicted in Fig 1. When opening the BiliNorm application, users have to fill in the following patient data: gestational age (weeks), date and time of birth, date and time of record, birth weight (g), TSB level (mg/dL or μmol/L) as well as risk factors (Fig 2A). The risk factors incorporated in this tool are adapted from the AAP guideline [9] and include ABO/Rhesus incompatibility, haemolysis (G6PD deficiency or spherocytosis), other illness (asphyxia, infection) and hypoalbuminemia (<30 mg/L). When 'no risk factors' is filled in, the case is considered as uncomplicated hyperbilirubinemia. When 'unknown risk factors' is filled in, the patient is categorised as having risk factors (other than those listed or not being able to check the risk factors). After filling in these data, the actual TSB level of the infant is shown in the TSB nomogram that displays the treatment thresholds of PT and the exchange transfusion over time. Advice for the caretaker on how to treat the infant is provided as follows: no treatment, start PT or immediately start intensive PT and consider exchange transfusion. Fig 2 shows an example of data entry in the BiliNorm application of a preterm infant born at 33 weeks of gestation. The infant was admitted to a neonatal unit with jaundice at day 2. Her birth weight was 2100 g. Blood test revealed ABO incompatibility and a TSB level of 17.2 mg/dL. After entering all data, the results showed both the treatment threshold lines of PT and the exchange transfusion for this patient. The recommendation for the paediatrician regarding this patient management is to start immediately with intensive PT and consider exchange transfusion.
Different nomograms are included in BiliNorm, one for infants with a gestational age >35 weeks and four for preterm infants. Infants born after <35 weeks of gestational age have different thresholds compared with term babies according to the new Indonesian National Guideline on Hyperbilirubinemia. In Indonesia, it is often difficult to determine the exact gestational age; therefore, guidelines for preterm babies are categorised into the following birth weight categories: �1000, 1001-1499, 1500-1999, and >1999 g.
In addition to the advice regarding the potential treatment for hyperbilirubinemia, information is provided about the risks of complications due to acute bilirubin encephalopathy (ABE). This is based on the modified bilirubin-induced neurological dysfunction-modified (BIND-M) scoring adapted from Radmacher et al. [10], which requires the examination of mental status, muscle tone, altered cry and altered gaze. The result is provided in the following four categories: 0, no ABE; 1-4, mild ABE; 5-6, moderate ABE; and >6, severe ABE. In this

PLOS ONE
Evaluation of a mobile application tool (BiliNorm) to improve care for newborns with hyperbilirubinemia case, the patient had mild hypotonia and a high-pitched cry, and thus, the BIND-M score of 2 was classified as mild ABE with a probable low risk of neurological complications (if appropriate treatment is provided timely).
Another feature of BiliNorm is the advice for follow-up examination in the outpatient clinic. This is based on the possible diagnosis of kernicterus spectrum disorder (KSD) and consists of a scoring system that includes the highest TSB level, the presence of risk factors, the findings of the neurological examination at first presentation and at follow-up, the presence of enamel dysplasia, the results of the auditory brainstem response (ABR) test and the magnetic resonance imaging findings [11]. The result is provided as one of the following four categories: definite kernicterus (10)(11)(12)(13)(14), probable kernicterus (6-9), possible kernicterus (3)(4)(5) and no kernicterus (0-2). This feature might help prepare for possible long-term complications of severe hyperbilirubinemia.
Communication to the patient's family is an often neglected and difficult issue for healthcare workers in low-and middle-income countries, especially in Indonesia. Therefore, Bili-Norm also provides an educational checklist on what should be told to the patient's family. The checklist is adopted and adapted from the National Institute for Health and Care Excellence (NICE) guideline on neonatal jaundice [12].
This investigation was an observational study conducted in two time epochs in two general district hospitals in East Java, i.e. Dr. Soetomo General Hospital, Surabaya, Indonesia, and Saiful Anwar General Hospital, Malang, Indonesia. Fig 3 shows the research algorithm.
The BiliNorm application was introduced to healthcare workers comprising midwives, paediatric residents and paediatricians in March 2019. The participants were asked to use the application after its introduction.
To evaluate how the BiliNorm application was perceived in practice, questionnaires were sent to healthcare workers who used the application through Google-form. The questionnaire was distributed through internal communication portal which consists of 150 residents and 53 NICU nurses. The questionnaire used in this study was adapted and developed from Davis' Technology Acceptance Model (TAM) [13]; it had the following four major components: 1. perceived usefulness, 2. perceived ease of use, 3. subjective norm and 4. intention to use Bili-Norm in the future. There were a total of 22 questions, with each question consisting of seven responses, resulting in a score ranging from 1 (strongly disagree) to 7 (strongly agree). The questionnaire used in this study is shown as S1 Data.
To examine whether the introduction of BiliNorm had an effect on the treatment of infants with hyperbilirubinemia, the medical records of all patients with neonatal hyperbilirubinemia admitted to the neonatal units were collected in both hospitals during a 6-month period before the introduction of BiliNorm (September 2018 to March 2019) and a 7-month period after its introduction (April to September 2019).
Data on gestational age, birth weight, birth date, risk factors, and TSB level were collected to determine the treatment that should have been provided to the patients based on the Indonesian Hyperbilirubinemia Guideline. Next, the actual treatment that was provided was compared with the treatment that was indicated by the guideline. All cases were divided into the following four groups: under-treatment, correct treatment, over-treatment and inappropriate treatment. Under-treatment implied that the infant did not receive any treatment despite the TSB level being above the PT threshold. Over-treatment implied that the infant received PT despite the TSB level being below the PT threshold. Correct treatment was defined as treatment according to the Indonesian Hyperbilirubinemia Guideline. Inappropriate treatment indicated that treatment was provided without any TSB measurement.

Data and statistical analysis
Data collected from the medical records were analysed using SPSS for Windows, Version 21 (IBM, Corp. Armonk, N.Y., USA). Pearson's chi-square test was used to calculate the p value of the proportion of infants within a gestational age category, birth weight category or having risk factors and treatment classifications for the pre-introduction period compared with the post-introduction period of BiliNorm. The p value of correct treatment, under-treatment, over-treatment and inappropriate treatment was calculated using Pearson's chi-square test for the comparisons of pre-introduction versus post-introduction of BiliNorm. Probability values <0.05 were considered to be statistically significant.

PLOS ONE
Evaluation of a mobile application tool (BiliNorm) to improve care for newborns with hyperbilirubinemia Table 1 shows all the questions and responses to the questionnaire. In total, 43 users of Bili-Norm returned the questionnaires, however the periodical changes of the number of healthcare workers involved in the communication portal made it hard to get the questionnaire's response rate. As much as 56% of the participants strongly agreed and another 28% agreed that the BiliNorm application helped in determining whether it is indicated to start PT in newborn infants with jaundice. The application also provided better quality of hyperbilirubinemia management (56% strongly agreed and 28% agreed). Furthermore, 81% of the respondents agreed or strongly agreed that it helped in providing better communication, information and education to the parents and family regarding hyperbilirubinemia and its effects. In the section 'perceived ease of use, 49% of them strongly agreed and another 23% agreed that it was easy to use BiliNorm. None of the respondents indicated that it was difficult to use. A small minority of 5% of the respondents reported slight disagreement on the three statements on clarity, flexibility and skillfulness required to use the application. For the questions regarding the subjective norm, 42% of them strongly agreed and another 28% agreed that their colleagues supposed that BiliNorm is important for the responder. Moreover, 42% of the respondents strongly agreed and 32% agreed to the intention to use BiliNorm in the near future.

Chart review
We collected 436 medical records from 255 infants with hyperbilirubinemia during the 6-month period before the introduction of BiliNorm and from 181 infants with hyperbilirubinemia during the 7-month period after its introduction. Fig 4 shows the demographic data of these infants. The proportion of infants receiving correct treatment was significantly higher after the introduction of BiliNorm than before its introduction (38% vs 51%; p < 0.006). There were few cases of under-and over-treatment, but this number was not statistically significant. The rate of treatment with PT without measuring the TSB level decreased from 14% to 7% (p = 0.024). Table 2 describes whether BiliNorm contributes to good results as implemented in the teaching hospital. BiliNorm had a good impact in providing suggestions to healthcare personnel as reported in the pre-post analysis, p = 0.016. The results indicated statistically significant differences between the correct treatment and inappropriate treatment rates before and after BiliNorm implementation. Fig 4 shows the results for infants with a gestational age above and below 35 weeks. There was no change in the incidence of correct treatment for infants >35 weeks of gestational age, ranging from 44% to 50% (p = 0.393), but the incidence increased for preterm infants <35 weeks of gestational age, ranging from 33% to 53% (p < 0.003). Over-and under-treatment rates did not change significantly in both age groups. The rate of treatment with PT without measuring the TSB level did not change significantly, the proportion being 9%-7% (p = 0.660) in infants >35 weeks of gestational age and decreasing from 18% to 7% (p < 0.014) in infants <35 weeks of gestational age. Out of 93 children who received correct treatment, only 1 child who received exchange transfusion.

Discussion
In this study, we developed a web-based application, BiliNorm, to be used on a smartphone to improve the usage of the newly developed Indonesian guideline on the diagnosis and treatment of hyperbilirubinemia. A survey conducted on a group of healthcare workers who were

PLOS ONE
Evaluation of a mobile application tool (BiliNorm) to improve care for newborns with hyperbilirubinemia introduced to the application indicated that it was perceived as helpful and easy to use. The majority of users indicated they would continue using the application. More than threefourths of the respondents agreed that BiliNorm helped them in the management of infants with hyperbilirubinemia. This application simplified the conventional approach of  hyperbilirubinemia management and made it easy for healthcare workers. An analysis of hospital charts on the use of PT before and after the introduction of BiliNorm indicated a higher correct usage rate of PT and a less usage rate of PT without measuring the TSB level, especially in preterm infants with a gestational age <35 weeks. Studies have demonstrated that the introduction of guidelines on how to diagnose and treat hyperbilirubinemia in newborn infants can help improve the care for newborn infants. However, the introduction of a guideline alone is not sufficient. Apparently, there is a need for campaigns to stimulate healthcare workers to use guidelines. Darling et al [14] explored the implementation of new guidelines formulated by the Canadian Pediatric Society in 100 hospitals in Canada and observed that 79 hospitals indicated to have implemented these guidelines. However, only 70% of these hospitals implemented measuring the TSB level before discharge, although this is recommended in the guidelines. Implementation of guidelines might help reduce the incidence of severe hyperbilirubinemia [15]. Sgro et al. found that after the implementation of the Canadian guidelines, the incidence of severe hyperbilirubinemia decreased from 1 in 2480 to 1 in 8352 live born infants [15].
Guidelines made for high-income countries (HIC) might not be suitable for low-and middle-income countries such as Indonesia because of the limited availability of well-equipped healthcare facilities. Therefore, a new Indonesian guideline on the diagnosis and treatment of hyperbilirubinemia was developed [7]. This new Indonesian Hyperbilirubinemia Guideline is an adoption of available international hyperbilirubinemia guidelines such as AAP guideline and NICE guideline. Previous hyperbilirubinemia guidelines from national and international institutions such as Indonesian Pediatric Society (IPS), Ministry of Health (MoH) dan World Health Organization (WHO) were evaluated and adopted into the new guideline in order to formulate a more suitable guideline to be applicated in Indonesia. The treatment thresholds to start PT are lower in this guideline than in the guidelines used in HIC. This was done to increase the safety margin, as daily control of the TSB level of a jaundiced infant is more difficult in Indonesia than in HIC. This study developed a web-based application to make the usage of this guideline easier. In a previous study, paediatricians indicated having difficulties to determine the existing guideline that hampered them to use it [3].
Comparable programs have been developed to assist healthcare workers to care for newborn infants with hyperbilirubinemia. Three of these programs, viz. BiliTool, BiliApp and the Northern California Neonatal Consortium (NCNC) app [2,16,17], are based on the AAP guideline of 2004 or on the NICE guideline. Most of these decision tools are applicable for infants born after �35 weeks of gestation, whereas others are also suitable for preterm infants. The existing decision tools only provide advices for the care of infants without risk factors. The user of these apps should make adjustments when risk factors are encountered. The BiliRecs program [18] has been designed for infants <35 weeks of gestational age, but it is unsuitable for infants with a postnatal age of <2 days.
In America, the AAP uses an application to improve compliance with guidelines made by an application known as Bilitool. However, unfortunately, Bilitool can be used only for babies with gestational age �35 weeks, and the reference guidelines used are from the AAP [18]. In UK, there have been efforts to improve compliance with NICE guidelines using a smartphonebased application known as BiliApp [17]. Unfortunately, although it can be used for almost all gestational age categories, the determination of gestational age in Indonesia is very difficult. This problem also exists in other low-middle-income countries. Moreover, bilirubin level is measured in the unit μmol/L, which is different from the unit used in Indonesia, i.e. mg/dL. This renders BiliApp unable to use, and hence, it would be difficult to implement it in Indonesia.
BiliNorm is different from the above-mentioned published tools. First, the treatment recommendations consider the risk factors if present. Second, the tool can be used for both term and preterm infants. Third, advices are provided regarding follow-up, based on the clinical signs of ABE in the postnatal period and on a KSD risk calculation. Fourth, BiliNorm provides information to parents about jaundice and treatment strategies. Finally, the application can be used on a smartphone. It is aimed for use in low-and middle-income countries and available in English and Bahasa languages as it is based on guidelines for Indonesia.
This study showed that after the introduction of our application in two hospitals, the incidence of correct treatment in all infants increased from 38% to 51%. Both over-and undertreatment rates of hyperbilirubinemia decreased slightly but did not reach statistical significance overall. However, >40% of the infants received treatment without good indication or did not receive treatment when indicated. The rate of use of PT without a TSB measurement decreased substantially, and it was 7% after the introduction of BiliNorm. Our study results indicate that particularly preterm infants <35 weeks of gestational age benefit the most from the introduction of BiliNorm; the rate of correct treatment increased from 33% to 53%, and that of inappropriate treatment decreased by more than half from 18% to 7%.
Limitations of BiliNorm might be caused by several factors such as; BiliNorm does not include clinical manifestation of the subject, where this factor influence clinical judgement of health care decision to do PT. It also does not involve social aspect as a consideration in hyperbilirubinemia treatment, which plays an important role especially in limited resource area. A lack of previous studies on this topic also limits theoretical foundations and comparisons related to the problem addressed in this study. More studies are required to understand why some healthcare workers either do not use or do not follow the available guidelines on the diagnosis and treatment of hyperbilirubinemia. Long-term studies are required to assess whether this application would result in improvements in the outcome of infants with hyperbilirubinemia and reductions in the incidence of severe hyperbilirubinemia and kernicterus. BiliNorm is only a decision support tool, and clinical decision is the responsibility of the clinician. As this study was conducted in two academic teaching hospitals, it might not reflect the entire implementation in daily practice where hyperbilirubinemia treatment is provided in various types of healthcare facilities that offer neonatal care.

Conclusions
A novel web-based application known as BiliNorm was developed to improve the management of hyperbilirubinemia in Indonesia. This decision support tool is based on the Indonesian National Guideline on Hyperbilirubinemia. The results of the questionnaire survey indicated that users found the tool user-friendly and helpful in the decision to start treatment/PT and indicated that they were planning to continue using it in the future. As the use of BiliNorm improved the rate of correct treatment of hyperbilirubinemia, expanding its implementation will improve the management of hyperbilirubinemia in Indonesia.

Disclaimer
The content of BiliNorm is designed to improve the management of jaundiced infants by qualified healthcare workers. BiliNorm is based on the Indonesian National Guideline on Hyperbilirubinemia. BiliNorm and its contents do not substitute professional medical advice, diagnosis or treatment. The use of all content of BiliNorm is entirely at your own risk. Authors of this article or the designer are not liable for any damages that may arise from the use of Bili-Norm. If you are dissatisfied with this disclaimer, then the only remedy is to stop using the application.